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SUMMARY 

A  model  of  an  association  memory,  the  EPAM  (Elementary 
Percelver  and  Memorizer),  is  a  learning  system,  a  computer 
simulation  of  human  verbal  learning  processes.  No  stored  infor¬ 
mation  in  this  memory  is  ever  physically  destroyed.  Yet  the 
behavior  which  we  normally  call  forgetting  occurs  because  of  a 
loss  of  access  (temporary  or  permanent)  to  information  stored,  in 
a  growing  net  of  associations.  In  this  system,  forgetting  occurs 
as  a  direct  consequence  of  normal  learning  processes  (i.e.,  for¬ 
getting  is  the  result  of  the  interference  of  items  later  learned 
with  items  learned  earlier)  without  the  postulation  of  a  separate 
mechanism.  Pwo  experiments  with  human  verbal  learning  are  dis¬ 
cussed,  and  the  interference  phenomena  are  explained  in  terms  of 


the  EPAM  model 
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FORGETTING  IN  AN  ASSOCIATION  MEMORY 


Though  the  phenomenon  of  forgetting  is  an  everyday  and 
commonplace  experience  of  human  intelligent  systems,  it  has  been 
inadequately  treated  by  theorists  of  intelligent  systems  (human 
and  computer). 

On  the  one  hand,  there  is  the  stark  behavioral  fact  that 
humans  who  were,  at  one  point  in  time,  able  to  respond  in  some 
particular  way  to  some  particular  stimuli  are,  at  a  later  time, 
no  longer  able  to  respond  to  those  stimuli.  On  the  other  hand, 
there  are  the  ever-intriguing  experiences,  shared  by  us  all,  that 
humans  are  able  to  recall  stimuli  whose  age-in-memory  is  of  the 
order  of  many  years  and  that  a  person's  techniques  of  association 
can  be  pressed  Into  the  service  of  retrieving  "long  lost"  Informa¬ 
tion  from  his  memory  (sometimes  quite  consciously  I ) .  Furthermore, 
some  modern  experiments  [i].  Involving  the  implantation  and 
stimulation  of  microelectrodes  in  the  brain,  have  suggested  that 
experiences  far  removed  in  time  can  be  evoked  in  great  informa¬ 
tional  detail  by  appropriate  stimulation  of  brain  tissue. 

All  of  this  suggests  the  possibility  that  forgetting  is 
caused  not  by  the  physical  destruction  of  information  in  the 
human  memory,  but  by  the  misplacing  —  the  more  or  less  temporary 
loss  of  access  to  —  information  in  memory.  This  possibility  has 
been  largely  unexplored. 

Instead,  theories  of  forgetting  have  specified  mechanisms 
for  thejdestruction  of  information  in  memory,  and  these  mechanisms 
are  treated  as  separate  from  the  mechanisms  which  account  for  the 
learning. 
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The  purpose  of  this  presentation  Is  to  explore  the  properties 
of  a  memory  In  which: 

(a)  Information  is  stored  in  association  with  encodings  of 
external  stimulus  configurations  and  with  (internal)  encodings 
already  stored  in  the  memory. 

(b)  no  stored  information  is  ever  destroyed  in  the  memory. 

(c)  what  we  normally  call  "forgetting"  occurs  as  a  direct 
consequence  of  the  learning  process  which  stores  information  in 
the  memory  (i.e.,  forgetting  is  the  result  of  the  interference 
of  items  later  learned  with  items  learned  .earlier). 

The  SPAM  Learning  System 

EPAM  (Elementary  Perceiver  and  Memorizer)  is  a  learning 
system  which  is  intended  to  be  a  simulation  of  human  verbal 
learning  processes.  The  information  processes  of  EPAM  are 
realized  as  computer  programs  in  the  list-language  IPL  V.  [2,  3] 

The  processes  perform  the  following  functions  (among  others): 

(a)  recognize  an  external  stimulus  as  one  about  which  some 
information  has  already  been  memorized. 

(b)  add  new  stimulus  items  to  the  memory  by  first  building 
discriminations  which  allow  the  new  item  to  be  distinguished  from 
stimuli  previously  learned. 

(c)  associate  (internally)  two  stored  items,  x  and  y,  by 
storing  with  x  some  cue  information  about  y. 

(d)  respond  to  an  external  stimulus  X  with  a  response  Y  by 
first  retrieving  the  cue  to  the  response  and  then  retrieving  the 
response  using  the  cue. 
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The  memory  structure  of  EPAM  is  the  discrimination  net. 

It  Is  a  tree  at  whose  terminal  points  are  stored  images  of  encodings 
of  external  stimuli.  At  the  nodes  of  the  tree  s.re  stored  tests 
which  examine  bits  of  the  encodings  of  stimuli.  The  image  of  a 
stimulus  is  retrieved  by  sorting  the  encoding  of  the  stimulus 
down  through  the  tests  of  the  net  to  the  appropriate  terminal. 

At  any  moment  in  the  learning  of  a  set  of  stimuli,  the  net  contains 
just  enough  tests  (roughly)  to  distinguish  the  stimuli  already 
learned . 

Information  about  a  new  stimulus  being  learned  is  added  to 
the  memory  by  first  growing  the  net  (i.e.,  adding  discriminations). 
The  new  item  is  sorted  to  a  terminal;  a  matching  process  produces 
a  difference  between  the  new  Item  and  the  one  stored  at  the 
terminal;  a  test  is  constructed  to  discriminate  on  this  difference; 
the  test  is  added  to  the  net,  thereby  creating  a  terminal  for  the 
storage  of  the  new  information. 

If  there  are  two  items,  x  and  y,  both  of  which  have  been 
learned  in  the  net,  association  of  y  as  the  response  to  x  is 
accomplished  by  storing  a  small  amount  of  the  y  information  with 
the  image  of  x  as  a  cue  to  the  associated  response.  The  system 
determines  the  amount  of  information  to  be  stored  as  a  cue  —  by 
trial  and  error,  as  that  quantity  of  information  just  sufficient 
to  retrieve  the  response  item  from  the  net  at  the  moment  the 
association  is  being  made. 

EPAM  responds  to  stimuli  by  recognizing  a  stimulus  (sorting 
it  in  the  net  to  its  terminal),  finding  there  the  cue  to  the 
response,  sorting  the  cue  In  the  net,  thereby  retrieving  the  image 
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of  the  response.  Not  all  the  time  will  the  response  be  retrieved 
by  this  process.  If  the  cue  lacks  information  which  the  dis¬ 
criminatory  tests  in  the  net  need,  then  the  response  image  is 
selected  randomly  from  the  undiscriminated  subset.  How  this 
situation  can  come  about  is  subsequently  discussed. 

This  has  been  a  very  incomplete  sketch  of  EPAM.  More  complete 
treatments  are  given  in  other  places  (Feigenbaum,  1959  [2]>  and 
1961  [3]). 

Interference  and  Forgetting 

How  can  it  happen  that  a  response  which  EPAM  was  able  to  make 
correctly  at  some  time  will,  at  a  later  time,  become  lost?  The 
answer  lies  in  the  fact  that  in  EPAM  the  "bond"  of  association  is 
indirect  (a  cue  is  sorted  in  the  net  of  items)  and  is  ever  vulner¬ 
able  to  interruption  by  further  learning. 

As  learning  (say,  in  verbal  learning  experiments)  proceeds, 
the  discrimination  net 'must  grow  to  encompass  the  new  items 
being  learned.  After  new  tests  and  branches  are  added  to  the  net 
at  some  end  point  (displacing  downward,  a  response  image  which  was 
located  there),  a  cue  which  at  some  previous  moment  of  association 
contained  enough  information  to  retrieve  the  response  image 
becomes  inadequate  (in  the  sense  that  it  lacks  the  information 
which  the  new  tests  wish  to  examine).  This  deficiency  will  not 
become  apparent  until  the  next  opportunity  arises  for  EPAM  to 
attempt  to  make  that  response.  In  an  attempt  to  respond  with  an 
Inadequate  cue,  a  wrong  response  image  may  be  selected  by  the 
random  mechanism  described  above  (i.e.,  a  response  generalization 
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will  occur).  If  the  error  can  be  detected  (e.g.,  from  feedback 
by  the  experimental  apparatus  in  an  experiment),  the  failure  can 
be  corrected  by  the  process  of  adding  more  information  about  the 
correct  response  to  the  cue. 

To  summarize,  learning  which  occurs  after  certain  associations 
are  made  may  interrupt  these  associations  by  growing  the  discrimi¬ 
nation  net  so  as  to  make  inadequate  the  association  cues  which  were 
previously  adequate. 

In  psychological  terms,  the  action  of  later  learning  in 
disturbing  associations  is  called  Interference.  The  SPAM  model 
gives  a  precise  explanation  of  the  phenomenon.  Forgetting  is  a 
feature  of  the  behavior  of  EPAM: 

(a)  because  access  to  responses  is  lost  in  a  growing  memory 
net  (but  the  loss  |need  be  only  temporary  if  the  difficulty  is 
detected  at  some  later  time). 

(b)  as  a  consequence  of  normal  learning  processes  without 
the  specification  of  a  separate  forgetting  mechanism. 

(c)  in  spite  of  the  fact  that  no  memorized  information  is 
actually  destroyed  (by  decay,  overwrite,  etc.). 

Some  Experimental  Evidence 

Consider  two  "classical"  verbal,  learning  experiments.  In 
the  first,  a  serial  list  of  items  is  presented  to  a  subject. 

When  presented  with  an  item  the  subject  must  respond  with  the 
next  item  on  the  list.  The  list  is  repeated  until  the  subject 
can  respond  correctly  to  all  items.  In  the  other  experiment, 
one  such  serial  list  is  learned;  then  a  second  list  is  learned; 
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then  a  retest  of  the  learning  of  the  first  list  is  made. 

A  feature  of  human  behavior  in  the  first  experiment  is  this: 
the  pattern  of  successes  and  failures  on  any  particular  item  is 
irregular:  a  few  successes,  then  a  failure,  then  further  successes, 

another  failure,  and  so  on.  This  phenomenon  has  been  termed 
oscillation  (by  Hull  M). 

Characteristic  of  behavior  in  the  second  experiment  is 
degradation  of  subjects'  ability  to  give  list  A  responses  in  the 
retest  after  the  interpolated  learning  of  the  list  B.  This 
phenomenon  has  been  termed  retroactive  inhibiticn. 

EPAM,  when  used  as  subject  in  the  same  experiments,  exhibits 
these  phenomena.  The  explanation  of  these  phenomena  in  EPAM 
terms  is  that  both  are  caused  by  the  interference  of  later  learning 
with  earlier  learning.  In  the  one-list  experiment,  this  is 
intralist  interference;  in  the  two-list  experiment,  interlist 
interference.  Both  phenomena  are  caused  by  one  and  the  same 
mechanism,  interference,  and  are  consequently  intimately  related.;1 
Oddly,  this  is  an  hypothesis  which  has  received  little  attention  j 
in  psychological  learning  theory.  i 

Conclusion 

We  have  presented  a  learning  system  which  exhibits  forgetting 
in  verbal  learning  situations,  even  though  no  "forgetting  mechanism" 
is  explicitly  postulated  and  no  destruction  of  information  in  the 
memory  takes  place.  This  is  not  a  hypothetical  system  but  a 
fully -realized  learning  machine. 

We  do  not  wish  to  argue  here  that  the  interference  process 
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described  above  is  the  only  process  of  human  forgetting.  We  are 
sympathetic  to  the  view  that  there  may  be  phenomena  of  forgetting 
which  go  beyond  interference  and  which  may  involve  an  assumption 
of  actual  information  loss.  What  we  have  demonstrated,  however, 
is  a  physical  learning  mechanism,  operating  without  the 
assumption  of  information  loss,  which  forgets  because  it  loses 
access  to  information  in  a  growing  memory  net  of  associations. 

The  demonstration  may  serve  to  clarify  and  sharpen  some  of  the 
issues  in  the  theory  of  forgetting  in  human  and  machine  learning 
systems. 
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